framework synthesize realistic world-consistent video
NVIDIA Novel vid2vid Framework Synthesizes Realistic World-Consistent Videos
A problem with video to video synthesis (vid2vid) is the technique's forgetfulness. Take the 2016 "Mannequin Challenge" viral video trend as an example: people remain frozen while a camera passes through them to capture the scene. Viewers would naturally be confused if the camera view returned to a previously captured person but their face appeared totally different -- as if they were wearing a magically transformative mask. This phenomenon has plagued vid2vid methods that generate video frames based only the information available in the immediately preceding frame(s). For AI researchers, achieving vid2vid temporal consistency over the entire rendered 3D world is a challenge.